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(57) A method for producing a compressed digital 
image from an input digital image is disclosed, wherein 
the compressed digital image is organized into layers 
having information that can be extracted in accordance 
with different desired viewing conditions and resolu- 
tions. The input digital image is decomposed to produce 
a plurality of subbands, each subband having a plurality 
of subband coefficients. The plurality of subband coef- 
ficients of each subband of the decomposed input digital 
image are quantized to produce a quantized output val- 
ue for each subband coefficient of each subband. At 
least one bit-plane is formed from the quantized output 
values of subband coefficients of each subband. Each 
bit-plane of each subband in at least one pass is entropy 
encoded to produce a compressed bit-stream corre- 
sponding to each pass, wherein each subband is entro- 
py encoded independently of the other subbands. A vis- 
ual quality table is provided that specifies a number of 
expected visual quality levels by providing selectable 
viewing conditions and resolutions for each expected 
visual quality level. A minimal set of passes and their 
corresponding compressed bit-streams that are neces- 
sary to satisfy the expected visual quality levels provid- 
ed in the visual quality table are identified, whereby a 
user can select different desired viewing conditions and 
resolutions for each compressed image. The com- 
pressed bit-streams corresponding to passes are then 
ordered into layers from the lowest expected visual qual- 
ity level to the highest expected visual quality level spec- 
ified in the visual quality table to produce a compressed 



digital image, wherein each layer includes the passes 
and their corresponding compressed bit-streams from 
the identified minimal set corresponding to the expected 
visual quality level that have not been included in any 
lower visual quality layers. 
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Description , 

[0001] This invention describes a method for produc- 
ing a compressed digital image organized into layers 
having information relating to different viewing condl- 5 
tions and resolutions. 

[0002] Subband or wavelet bit-plane coding is being 
used in the proposed JPEG2000 standard, as described 
in ISO/IEC JTC1/SC29 WG1 N1646. JPEG2000 Part I 
Final Committee Draft, Version 1.0, March 2000. The 10 
JPEG2000 encoder decomposes the image into a hier- 
archy of resolutions and the compressed data corre- 
sponding to a resolution is further divided into a number 
of quality layers. At any resolution, adding more layers 
to the compressed bit-stream generally improves the is 
visual quality of the image reconstructed at that resolu- 
tion and at higher resolutions. The JPEG2000 standard 
provides a very flexible framework for organizing and 
ordering the compressed bit-stream. For example, a 
compressed bit-stream may be resolution-layer-compo- 20 
nent-position progressive or layer-resolution-compo- 
nent-position progressive. Such bit-streams are also re- 
ferred to as resolution-scalable and quality-scalable, re- 
spectively. It is up to the individual JPEG2000 encoder 
to arrange the compressed bit-stream in a manner suit- 2s 
able for the intended application. 
[0003] If the compressed bit-stream is arranged in a 
resolution-layer-component-position progressive man- 
ner, henceforth referred to as resolution-progressive, all 
the layers corresponding to a lower resolution image ap- 30 
pear in the bit-stream earlier than any layers corre- 
sponding to higher resolutions. On the other hand, if the 
compressed bit-stream is arranged in a layer-resolution- 
component-position progressive manner, henceforth re- 
ferred to as layer-progressive, layers with lower indices 35 
from all resolution levels appear in the bit-stream earlier 
than any layers with higher indices. Thus, it is not pos- 
sible for the compressed bit-stream to be resolution-pro- 
gressive and layer-progressive at the same time. But in 
some applications, a combination of these two modes *o 
may be desirable. 

[0004] For example, consider a digital camera having 
four visual quality levels. Suppose that the images are 
captured at a resolution of 1536 by 1024 pixels, but for 
the two lowest visual quality levels, the images are 4$ 
stored at the next lower resolution, 768 by 512 pixels. 
Consider a usage scenario in which the image Is com- 
pressed at the highest visual quality level, but due to 
memory constraints It is necessary to truncate the com- 
pressed bit-stream so that the image is stored at the low- so 
est visual quality level. If the original compressed bit- 
stream had been arranged in a layer-progressive man- 
ner, the truncated bit-stream would still contain some 
layers from the higher (1536 by 1024) resolution. This 
is a waste of memory space because at the lowest visual ss 
quality level, the image will be stored at the 768 by 512 
resolution. Similarly, suppose that the compressed bit- 
stream corresponding to the highest visual quality level 



is arranged in a resolution-progressive manner, and is 
then truncated to the next lower visual quality level at 
the same resolution. In this case, the truncated bit- 
stream may contain layers from the 768 by 512 resolu- 
tion that are not necessary to achieve the desired visual 
quality level. 

[0005] Also consider the situation where the digital 
camera stores many images on a flash memory card, 
and includes an image display, such as a color liquid 
crystal display (LCD), with a limited dynamic range. The 
LCD displays several "thumbnail size" (for example, 1 92 
by 128 resolution) images at the same time to allow the 
user to locate an image of interest It then displays a 
single "screennail" size (for example 384 by 256 reso- 
lution) image of one of the selected thumbnail images. 
In this situation, if the file is arranged in a layer-progres- 
sive manner, It will take time to skip over some layers 
from the higher (1536 by 1024) resolution In order to lo- 
cate just the data needed for the lower resolution imag- 
es. If the data is arranged in a resolution-progressive 
manner, it will again take time to skip over some of the 
layers from the 192 by 128 and 384 by 256 resolution 
levels that are not needed to provide an acceptable im- 
age on the low dynamic range camera LCD. As a result, 
the time required to decode the images will be slow, and 
the display will not be as responsive to user commands 
to display a new image as would be desirable. 
[0006] Accordingly, an object of the present invention 
is to provide a method for the formation of layers of a 
compressed JPEG2000 bit-stream in such a manner 
that the bit-stream provides partial resolution scalability 
within a layer-progressive ordering. 
[0007] This object is achieved by a method for pro- 
ducing a compressed digital image from an input digital 
image, wherein the compressed digital image is organ- 
ized into layers having information that can be extracted 
in accordance with different desired viewing conditions 
and resolutions, comprising the steps of: 

(a) decomposing the input digital image to produce 
a plurality of subbands, each subband having a plu- 
rality of subband coefficients; 

(b) quantizing the plurality of subband coefficients 
of each subband of the decomposed input digital 
image to produce a quantized output value for each 
subband coefficient of each subband; 

(c) forming at least one bit-plane from the quantized 
output values of subband coefficients of each sub- 
band; 

(d) entropy encoding each bit-plane of each sub- 
band in at least one pass to produce a compressed 
bit-stream corresponding to each pass, wherein 
each subband is entropy encoded independently of 
the other subbands; 

(e) providing a visual quality table that specifies a 
number of expected visual quality levels by provid- 
ing selectable viewing conditions and resolutions 
for each expected visual quality level; 
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(f) identifying a minimal set of passes and their cor- 
responding compressed bit-streams that are nec- 
essary to satisfy the expected visual quality levels 
provided in the visual quality table, whereby a user 
can select different desired viewing conditions and 
resolutions for each compressed image; and 

(g) ordering the compressed bit-streams corre- 
sponding to passes into layers from the lowest ex- 
pected visual quality level to the highest expected 
visual quality level specified in the visual quality ta- 
ble to produce a compressed digital image, wherein 
each layer includes the passes and their corre- 
sponding compressed bit-streams from the identi- 
fied minimal set corresponding to the expected vis- 
ual quality level that have not been included in any 
lower visual quality layers. 

[0008] The present invention provides an advantage 
in that the compressed bit-stream corresponding to the 
highest visual quality level can be truncated to achieve 
any other visual quality level. 

[0009] The present invention also provides an advan- 
tage in that the truncated bit-stream contains only the 
layers that are necessary to represent the image at the 
resolution intended for that visual quality level. 
[001 0] In describing the preferred embodiments of the 
invention reference will be made to the series of figures 
and drawings briefly described below. 

FIG. 1 shows a flow chart of an image encoder ac- 
cording to the present invention; 
FIG. 2 shows a resolution hierarchy created by a 3 
level octave subband decomposition; 
FIG. 3 shows a block diagram of the codeblock com- 
pression unit of FIG. 1; 

FIGS. 4A and 4B show graphs of the decision 
thresholds and reconstruction levels for step-sizes 
of A and 2 A, respectively, for a uniform scalar quan- 
tizer with a dead-zone; 

FIG. 5 shows a flow chart of the layer formation 
and ordering decision unit" of FIG. 1; 
FIG. 6 shows a flow chart of another embodiment 
of the layer formation and ordering decision unit" 
of FIG. 1; 

FIG. 7 shows a flow chart for reconfiguring a com- 
pressed bit-stream to provide partial resolution 
scalability within a layer-progressive ordering; 
FIG. 8 shows a block diagram of a digital camera 
system including a digital camera and a digital com- 
puter which can utilize the present invention; and 
FIG. 9 shows the back of the digital camera depict- 
ed in FIG. 8, including the liquid crystal display 
(LCD). 

[0011] There may be additional structures described 
in the foregoing application that are not depicted on one 
of the described drawings. In the event such a structure 
is described, but not depicted in a drawing, the absence 



of such a drawing should not be considered as an omis- 
sion of such design from the specification. 
[0012] The present invention relates to compression 
of a digital image. Although there are other techniques 
s well known in the art, the present invention will be de- 
scribed with respect to the technique set forth in the 
JPEG2000 image compression standard. Because the 
proposed JPEG2000 image compression standard 
specifies how the decoder shall interpret a compressed 
10 bit-stream, there are certain inherent restrictions on any 
JPEG2000 encoder. For example, in Part I of the stand- 
ard, only certain wavelet filters can be used. The entropy 
coder is also fixed. These methods are described in ISO/ 
IEC JTC1/SC29 WG1 N1646, JPEG2000 Part I Final 
is Committee Draft, Version 1 .0, March 2000. Hence, the 
present description will be directed in particular to at- 
tributes forming part of, or cooperating more directly 
with, the algorithm in accordance with the present in- 
vention. Attributes not specifically shown or described 
herein may be selected from those described in ISO/IEC 
JTC1/SC29 WG1 N1646, JPEG2000 Part I Final com- 
mittee Draft, Version 1.0, March 2000, or otherwise 
known in the art. in the following description, a preferred 
embodiment of the present invention would ordinarily be 
implemented as a software program, although those 
skilled in the art will readily recognize that the equivalent 
of such software may also be constructed in hardware. 
Given the system and methodology as described in the 
following materials, all such software implementation 
needed for practice of the invention is conventional and 
within the ordinary skill in such arts. If the invention is 
implemented as a computer program, the program may 
be stored in conventional computer readable storage 
medium, which may comprise, for example; magnetic 
storage media such as a magnetic disk (such as a floppy 
disk) or magnetic tape; optical storage media such as 
an optical disc, optical tape, or machine readable bar 
code; solid state electronic storage devices such as ran- 
dom access memory (RAM), or read only memory 
(ROM); or any other physical device or medium em- 
ployed to store a computer program. 
[0013] Reference will now be made in detail to the 
present preferred embodiment of the invention, an ex- 
ample of which is illustrated in the accompanying draw- 
ings. While the invention will be described in connection 
with a preferred embodiment, it will be understood that 
it is not intended to limit the invention to that embodi- 
ment. On the contrary, it is intended to cover all alterna- 
tives, modifications, and equivalents as may be included 
within the spirit and scope of the invention defined in the 
appended claims. 

[001 4] A flow chart of a JPEG2000 image encoder ac- 
cording to the present invention is shown in FIG. 1. A 
digital image (101) undergoes subband decomposition 
(102) by the analysis filters to produce an image repre- 
sentation in terms of subband coefficients (103). If the 
image has multiple components (for example, RGB), a 
luminance-chrominance transformation can be applied 
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to convert it to a YCbjCr representation, before the sub- 
band decomposition step (102). Also, it is possible to 
divide each component of the image into multiple tiles. 
But in this preferred embodiment, only a single tile con- 
sisting of the entire image is used. The subband coeffi- 5 
cients (103) are partitioned into rectangular blocks by 
the codeblock partitioning unit (104) to produce one or 
more codeb locks (105). Those skilled in the art would 
appreciate that partitioning of the subband coefficients 
is not necessary if only a single codeblock is used. Each to 
codeblock is compressed by the codeblock compres- 
sion unit (106) using the appropriate quantizer step-size 
(109) to produce a compressed codeblock (107) and a 
byte-count table (108). For each codeblock, the com- 
pressed bit-stream (107) and the byte-count table, de- is 
noted by TableB (108), are fed to a layer formation and 
ordering decision unit (111). The other inputs to the layer 
formation and decision unit (1 1 1 ) are the quantizer step- 
size (109) used to quantize that codeblock and a table, 
denoted by TableQ (110), containing information about 20 
the desired visual quality levels, intended resolution, 
and viewing condition parameters for the respective vis- 
ual quality levels. For each codeblock, the layer forma- 
tion and ordering decision unit (111) determines how 
many coding passes should be included in each layer, 25 
and produces a layered compressed codeblock (112) 
and a table, denoted by TableL (113). TableL stores in- 
formation about the number of coding passes and the 
corresponding bytes in each layer for that codeblock. 
The layer formation and ordering decision unit (111) also 30 
specifies that the overall bit-stream is to be arranged in 
a layer-progressive manner. The layered compressed 
codeblocks (112), TableL (113), and the ordering infor- 
mation are fed to the JPEG2000 bit-stream organizer 
(114) to produce an encoded digital image (115) that is 35 
JPEG2000 compliant The master table generator (1 1 6) 
generates TableML (117) whose I th entry specifies the 
number of bytes required to represent the compressed 
image data corresponding to the first i layers. This infor- 
mation is also contained in the compressed bit-stream, *o 
but in some applications it may be advantageous to 
store the information separately so that it is not neces- 
sary to parse the bit-stream for the information. 
[0015] The blocks in FIG. 1 will now be described in 
greater detail. The image undergoes subband decom- *s 
position by analysis filters (102). A single level of de- 
composition splits the image into 4 subbands, denoted 
by LL, HL, LH, and HH. The first letter indicates the type 
of filtering employed in the horizontal direction. The sec- 
ond letter indicates the type of filtering employed in the so 
vertical direction. For example, band HL is the result of 
high-pass filtering in the horizontal direction and low- 
pass filtering in the vertical direction. In an octave de- 
composition, the LL band is split recursively. In the pre- 
ferred embodiment, an R level octave decomposition is 55 
used. FIG. 2 shows an example of a 3 level octave de- 
composition. The subbands are numbered from the low- 
est frequency to the highest frequency. An R level de- 



composition generates (R+1) resolutions and (3R+1) 
subbands, denoted by S 0 , S 1 S 3R . An image at reso- 
lution r (0 £ r £ R) can be reconstructed from subbands 
S 0 , S 1 ,...,S 3r If the original image is of size 1024 by 
1024, and the decomposition is 3 level octave, images 
at resolutions 0, 1; 2 and 3 have sizes 128 by 128, 256 
by 256, 512 by 512, and 1024 by 1024, respectively. 
[0016] The codeblock partitioning unit (104) shown in 
FIG. 1 partitions each subband into a number of rectan- 
gular codeblocks. The codeblock compression unit 
(106) is shown in greater detail in FIG. 3. Each code- 
block is quantized with a scalar quantizer (201) to pro- 
duce a sign-magnitude representation of the indices of 
quantized coefficients (202). Preferably, a uniform sca- 
lar quantizer with a dead-zone is used. The decision 
thresholds and reconstruction levels for this quantizer 
are shown in FIGS. 4A and 4B. FIG. 4A shows the de- 
cision thresholds and reconstruction levels for a step- 
size of A; FIG. 4B shows the decision thresholds and 
reconstruction levels for a step-size of 2 A. In a preferred 
embodiment, the reconstruction levels are always at the 
center of the quantization interval. But those skilled in 
the art will recognize that this is not necessary. For ex- 
ample, the reconstruction levels can be biased towards 
zero. The same base quantizer step-size is used for all 
the codeblocks in a given subband. Let the step-size for 
subband i be Aj. It should be noted that the maximum 
quantization error, denoted by E^, is (A| / 2), except 
for the zero bin which has a maximum quantization error 
of Aj. If the subband analysis and synthesis filters are 
reversible (R. Calderbank, I. Daubechies, W. Sweldens, 
and B.-L. Yeo, "Wavelet Transform that Maps integers 
to Integers," Applied and Computational Harmonic 
Analysis, vol. 5, no. 3, pp. 332-369, 1998), the quanti- 
zation step may be entirely absent 
[0017] Suppose that the codeblock being processed 
is from subband i. Then, the samples from the block are 
quantized with a uniform scalar quantizer with step size 
A] as described above. Suppose that the magnitude of 
the indices of quantized coefficients is represented by 
a fixed precision of Aj bits. Let the bits be indexed as 
1,2,...,Aj, where index 1 corresponds to the most signif- 
icant bit (MSB) and A) corresponds to the least signifi- 
cant bit (LSB). The k^ bit-plane for the codeblock con- 
sists of the kft bit from the magnitude representation of 
all the indices of quantized coefficients from that code- 
block. One interesting property of the scalar quantizer 
being used is that discarding, or zeroing out the k least 
significant bits from the magnitude representation of the 
index of a quantized coefficient from subband 1 is equiv- 
alent to scalar quantization of that coefficient with a step- 
size of 2 k Aj. Thus, if the compressed bit-stream corre- 
sponding to the codeblock is truncated so that the data 
corresponding to the last k bit-planes is discarded, it is 
possible to reconstruct a more coarsely quantized ver- 
sion of the codeblock. This is known as the embedding 
property. It should be noted that If the last k bit-planes 
of the magnitude representation of a quantized coeffi- 
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dent are dropped, for, reconstruction at the decoder, the 
reconstruction levels for the quantizer with a step-size 
of 2 k Aj are used. 

[0018] For the purpose of entropy coding, a bit-plane 
for a codeblock is said to be significant if any of the pre- 5 
vious bH-planes were significant or the bit-plane has at 
least one non-zero bit. The entropy encoder (203) codes 
each bit-plane for the codeblock in one or more coding 
passes. For example, the most significant bit-plane is 
encoded using a single coding pass. The rest of the bit- 10 
planes for the codeblock are encoded using three cod- 
ing passes. In JPEG2000, the MQ arithmetic coder is 
used as the entropy coder. The table generation unit 
(204) generates a byte-count table (108) for each code- 
block. The m" 1 entry in the table corresponds to the is 
number of bytes needed to include coding passes 

1,2 m of the codeblock in the bit-stream. 

[001 9] The layer formation and ordering decision unit 
(111) determines the number of coding passes to be in- 
cluded in each layer so that the visual quality criteria as 20 
specified by TableQ (110) are met TableQ (110) stores 
information about the desired visual quality levels. The 
1 st column entry of the j* row, TableQ[J][1]. specifies the 
resolution at which the image at visual quality level j will 
be stored. The 2 nd column entry, TableQ[j][2], specifies 2s 
the viewing condition parameters applicable for visual 
quality level j. As will be described shortly, the viewing 
condition parameters determine the effective quantizer 
step-size to be used in each subband and thus, deter- 
mine the visual quality level for the reconstructed image. 30 
The number of layers, L, for each codeblock is equal to 
the number of visual quality levels in TableQ. 
[0020] For a given visual quality level and the corre- 
sponding viewing condition parameters, for each sub- 
band, the layer formation and ordering decision unit 35 
(111 ) calculates the quantizer step-size that will result in 
just noticeable distortion in the reconstructed image. 
This is accomplished by using the two-dimensional Con- 
trast Sensitivity Function (CSF) of the human visual sys- 
tem (HVS). The CSF model described in Jones, "Com- 40 
pa native study of wavelet and DCT decomposition with 
equivalent quantization and encoding strategies for 
medical images", Proc. SPIE Medical Imaging *95. vol. 
2431, pp. 571-682, which Is incorporated herein by ref- 
erence, models the sensitivity of the human visual sys- 45 
tern as a function of the two-dimensional (2-D) spatial 
frequency, and it depends on a number of parameters, 
such as viewing distance, light level, color, image size, 
eccentricity, noise level of the display, and so forth 
These parameters will be referred to as viewing condi- 50 
tion parameters. The frequency dependence of the CSF 
is commonly represented using cycles/degree of visual 
subtense. The CSF can be mapped to other units, such 
as cycles/mm, for a given viewing distance (that is, the 
distance from the observer to the displayed image). 55 
[0021] The 2-D CSF value for subband i is denoted 
by CSF(F h VC), where VC refers to the viewing condition 
parameters which include the intended viewing dis- 



tance, the ambient light level, the display noise level, 
and the dots per inch for the display. F, represents the 
2-D spatial frequency (in cycles/mm) associated with 
subband i. In a preferred embodiment, F, is chosen to 
be the center of the frequency range nominally associ- 
ated with subband i. As described in the Jones, if sub- 
band I is quantized with a uniform scalar quantizer hav- 
ing a dead-zone, the step-size Q f (VC) that results in just 
noticeable distortion in the reconstructed image for 
viewing condition parameters VC is 



^ VVW/ ^C x MTF(F,) X G, x CSF(F,, VC)' 

where MTF(F,) is the display MTF at frequency >,, C is 
the contrast per code-value of the display device, and 
Gj is the gain factor that represents the change in con- 
trast for the reconstructed image for one code-value 
change in a coefficient of subband i. The gain factor de- 
pends on the level and orientation of the subband, as 
well as the subband synthesis filters. Compared to the 
paper by Jones et al, a factor of 0.5 is missing from the 
denominator. This is due to the fact that for uniform sca- 
lar quantizer with a dead-zone, the maximum possible 
distortion, E max , is equal to the step-size, as opposed to 
half the step-size for a uniform scalar quantizer in the 
absence of a dead-zone. 

[0022] The viewing condition parameters are pre- 
specified for each visual quality level. Images stored at 
higher visual quality levels are generally viewed under 
more exacting conditions, for example smaller intended 
viewing distances. In some cases, the viewing condition 
parameters may not correspond exactly to the actual 
viewing conditions. For example, suppose that a digital 
camera captures images at a high enough resolution to 
make 5 by 7 inch prints. Furthermore, suppose that 
three visual quality levels, good, better and best, are of- 
fered at that resolution. If the prints are to be viewed at 
a viewing distance of 10 inches, the intended viewing 
distance for the best quality is specified to be 1 0 inches. 
The viewing distances for the remaining two visual qual- 
ity levels are set to higher values, for example, 1 2 inches 
for better visual quality and 16 inches for good visual 
quality. All the remaining viewing condition parameters 
are the same for the three quality levels. This results in 
higher quantizer step-sizes for the good and better vis- 
ual quality levels, resulting in suprathreshold quantiza- 
tion artifacts in the reconstructed images. The visual dis- 
tortion for the image reconstructed at the best visual 
quality level is just noticeable. Similarly the display noise 
levels can be increased beyond the actual value while 
keeping the other viewing condition parameters the 
same, to provide lower visual quality levels. 
[0023] Now consider a codeblock from subband i 
quantized with a step-size A|. Suppose that m coding 
passes for the codeblock are included in the com- 
pressed bit-stream, and the codeblock is reconstructed 
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from the m coding passes. Let E max be the maximum 
absolute difference between the original codeblock and 
the reconstructed codeblock. The codeblock can be 
thought of as being quantized by a uniform scalar quan- 
tizer with a dead-zone having a step-size of E^. In this 
case the effective step-size is E,^. Assume that for the 
codeblock from subband i, Pq_^ coding passes have 
been included in coding layers 1,2 (j-1). and it is nec- 
essary to determine the number of coding passes to be 
included in layer j so that the visual quality level j is at- 
tained (or exceeded) by including the first j layers in the 
bit-stream. The quantizer step-size Q f (VCj) that results 
in just noticeable distortion in the reconstructed image 
is calculated. Here VCj refers to the viewing condition 
parameters at visual quality level j. Then, for coding 
passes Pq.!), Pq-i) + 1 . ••• and so forth, the effective quan- 
tizer step-size is calculated. Suppose that at pass m, the 
effective quantizer step-size becomes less than or equal 
to Qi(VCj), then Pj is set to m. This means that (m -Pq_i)) 
additional coding passes are required in the layer of 
that codeblock. This process is repeated for each code- 
block from each subband, and for each visual quality 
level. 

[0024] It is possible that some layers in a codeblock 
may contain no data. For example, consider that an im- 
age is to be stored at visual quality level 3, and let the 
corresponding resolution be r. Now, suppose for a code- 
block from subband i, the number of coding passes to 
be included in layer 3 is to be determined. As described 
previously, if i is greater than 3r, subband i is not neces- 
sary to reconstruct the image at resolution r. So no data 
is included in layer 3 of codeblocks belonging to sub- 
band i. 

[0025] A flow chart for the layer formation and order- 
ing decision unit (1 1 1 ) is shown in FIG. 5. Let the number 
of layers for the codeblock be L, and the total number 
of coding passes for the codeblock be P. The inputs to 
the layer formation and ordering decision unit are: 1 ) 
The visual quality table, TableQ (110), with L entries, 2) 
the original codeblock (105), 3) the compressed bit- 
stream corresponding to that codeblock (107), and 4) 
the byte-count table, denoted by TableB (108) for the 
codeblock. The layer formation and ordering unit (111) 
generates a table, denoted by TableL (113), that has L 
rows and 2 columns. The 1 st entry from row j denotes 
the number of coding passes belonging to layer j, and 
the 2 nd entry indicates the number of bytes in layer j of 
that codeblock. 

[0026] The initializer unit (501 ) initializes j, m, and the 
number of cumulative passes, CP, to zero. It also Initial- 
izes the maximum absolute error, E max , to the maximum 
absolute value of the indices of quantized coefficients 
for that codeblock. In step (502), j is incremented by 1 . 
Then, the comparison unit (503) compares j against the 
number of layers, L. If j is greater than L, ail the layers 
have been formed, and the process is stopped after writ- 
ing out TableL (113). Otherwise, the process is contin- 
ued. In step (504), the resolution r is set to TableQ[j][1]. 



A second comparison unit (505) compares the index of 
the subband to which the codeblock belongs, i, with 3r. 
If i is greater than 3r, subband i is not needed to recon- 
struct the image at resolution r. The flow-control is 
5 passed to step (510) and the p 1 row of TableL is set to 
0. This signifies that no coding passes from the code- 
block are included in layer j. If i is less than or equal to 
3r, another comparison unit (506) compares with 
the quantizer step-size, QjfVCj), that produces just no- 
lo ticeable distortion for viewing condition parameters cor- 
responding to visual quality level j. If E,^ is less than 
or equal to Qj(VCj), the flow-control passes to step 
(510), and all coding passes up to and including m are 
included in layer j. Otherwise, m is compared against 
is the total number of passes, P (507). If m is greater than 
or equal to P, the flow-control skips to step (510). Oth- 
erwise, m is incremented by 1 (508). Then, the code- 
block is reconstructed by using compressed data corre- 
sponding to the first m coding passes, and the maximum 
absolute difference, Em^, between the original code- 
block and the reconstructed codeblock is found (509). 
After this, the flow-control returns to step (506). In step 
(510), TableL[j][1] is set to (m - CP) and TableLfflP] is 
set to (TableBfm] - TabieB[CP]). Also, the number of cu- 
mulative passes is set to m. Then, the flow-control re- 
turns to step (502). Thus, steps 506 through 509 have 
the effect of identifying a minimal set of passes and their 
corresponding compressed bit-streams that are neces- 
sary to satisfy each expected visual quality level provid- 
ed in the visual quality table, TableQ (110). 
[0027] It should be noted that the step-size used to 
quantize the codeblock should be sufficiently small so 
that when all the coding passes for the codeblock are 
included in the bit-stream, the maximum quality level 
specified in the visual quality table, TableQ (110), can 
be achieved or exceeded. 

[0028] Another embodiment of the layer formation 
and ordering decision unit Is shown in FIG. 6, where an 
additional constraint is placed on the formation of the 
layers. The constraint is that the layer boundaries for a 
block must coincide with the bit-plane boundaries. As 
discussed previously, let the magnitudes of the indices 
of quantized codeblock coefficients, quantized with a 
step-size of A, be represented by a fixed precision of A 
bits. Let the bits be indexed as 1.....A, with index 1 rep- 
resenting the MSB. Now suppose that the k least signif- 
icant bit-planes of the codeblock are discarded. Then, 
the effective quantizer step-size for the codeblock is (2 k 
A). Instead of calculating the maximum absolute error, 
E max , between the original codeblock and the recon- 
structed codeblock as done previously, E,^ is set equal 
to2 k A. 

[0029] In the alternate embodiment, the initializer unit 
(601 ) also initializes k to 0. Steps 602 - 606 are identical 
to steps 502 - 506. In step (607), k is compared with A, 
the total number of bit-planes for the codeblock. If k is 
greater than or equal to A the flow-control passes to 
step (610). Otherwise, in step (608), k is incremented 
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by 1 , and m is updated so that m represents the number 
of coding passes needed to represent the first k bit- 
planes. In step (609), the effective step-size corre- 
sponding to retaining only the first k bit-planes, (2^-*) 
A), is calculated, and E max is set to this value. Step 610 
is identical to steps 510. Thus, steps 606 through 609 
have the effect of identifying a minimal set of passes 
and their corresponding compressed bit-streams that 
are necessary to satisfy each expected visual quality 
level provided in the visual quality table, TabteQ (110). 
[0030] Another embodiment of the invention is shown 
in FIG. 7. A compressed bit-stream (701) produced by 
a JPEG2000 encoder is passed through a JPEG2000 
bit-stream parser (718) to produce compressed bit- 
stream corresponding to each codeblock (719). The bit- 
stream parser also extracts information about quantizer 
step-sizes (709). Each compressed codeblock bit- 
stream is passed through an entropy decoder (702) to 
produce quantized subband coefficients (703). Steps 
704 - 717 are exactly identical to the corresponding 
steps 104 - 11 7. It should be noted that if the base quan- 
tizer step-sizes used to produce the original JPEG2000 
bit-stream are coarse, it may not be possible to achieve 
all the visual quality levels from the visual quality table 
(710). 

[0031 ] The invention can be incorporated into a digital 
still camera 300 shown in FiG. 8. The digital still camera 
300 produces digital images that are stored on a remov- 
able memory card 330. The digital still camera 300 in- 
cludes a zoom lens 312 having zoom and focus motor 
drives 310 and an adjustable aperture and shutter (not 
shown). The user composes the image using an optical 
viewfinder 311 and a zoom lens control switch 372 
(shown in FIG. 9). The zoom lens 312 focuses light from 
a scene (not shown) on an image sensor 314, for exam- 
ple, a single-chip color CCD image sensor, using the 
well-known Bayer color filter pattern. The image sensor 
314 is controlled by clock drivers 306. The zoom and 
focus motors 310 and the clock drivers 306 are control- 
led by control signals supplied by a control processor 
and timing generator circuit 304. When the user de- 
presses a shutter button 374 to take a picture, the con- 
trol processor and timing generator 304 receives inputs 
from autofocus and autoexposure detectors 308 and 
controls a flash 302. The analog output signal from the 
image sensor 314 is amplified and converted to digital 
data by an analog signal processing (ASP) and analog- 
to-digital (A/D) converter circuit 316. The digital data is 
stored in a DRAM buffer memory 318 and subsequently 
processed by a processor 320 controlled by the 
firmware stored in a firmware memory 328, which can 
be flash EPROM memory. 

[0032] The processed digital image file is provided to 
a memory card interface 324 which stores the digital im- 
age file on the removable memory card 330. The proc- 
essor 320 performs color interpolation followed by color 
and tone correction, in order to produce rendered sRGB 
image data. The rendered sRGB image data is then 



compressed using the methods described earlier and 
stored as a JPEG 2000 image file on the removable 
memory card 330. The processor 320 also creates a 
"screennaii" size image that is stored in RAM memory 
s 326 and supplied to the color LCD image display 332, 
which displays the captured image for the user to review. 
[0033] The graphical user interface displayed on the 
color LCD image display 332 is controlled by user inter- 
face switches 380, 381, 382, 383, 384, and 386 shown 
10 in FIG. 9. After a series of images has been taken with 
the switch 386 in the capture position, the switch 386 
can be moved to the review position so that the captured 
pictures can be reviewed on the color LCD image dis- 
play 332. 

[0034] Suppose that in our earlier example, the digital 
still camera 300 has 2 visual quality levels (for example, 
good and best) at the 192 by 128 (thumbnail) resolution 
and the 384 by 256 (screennaii) resolution. Further- 
more, the digital still camera 300 has three visual quality 
levels (good, better, and best) each at the 768 by 512 
(email) resolution and 1536 by 1024 (print) resolution. 
Then, the compressed image generated by a 
JPEG2000 encoder according to the preferred embod- 
iment has 1 0 quality layers. The JPEG2000 decoder can 
decode just the first quality layer to display a "good" 1 92 
by 128 thumbnail image. Similarly, the JPEG2000 de- 
coder can decode the first 3 quality layers to obtain a 
"best" 384 by 256 image and so on. 
[0035] After a series of images have been taken and 
stored on the removable memory card 330, the remov- 
able memory card 330 can be inserted into a memory 
card reader (not shown) in the user's host computer 340. 
Alternatively, an interface cable 342 can be used to con- 
nect between the host interface 322 in the digital still 
camera 300 and the corresponding camera interface in 
the host computer 340. The interface cable 342 may 
conform to, for example, the well-know universal serial 
bus (USB) interface specification. 
[0036] Initially, the user may wish to download "better" 
quality 768 by 512 resolution images. In that case, only 
the first 6 quality layers are transferred from the digital 
still camera 300 to the host computer 340. This may be 
sufficient resolution for emailing the image. The inven- 
tion reduces the download time by only transferring the 
quality layers necessary to obtain a user-specified qual- 
ity and resolution. The user may decide that he would 
like to print the image at the highest possible quality. In 
that case, the remaining 4 quality layers are transferred 
from the digital still camera 300 to the host computer 
340. 



Claims 

1 . A method for producing a compressed digital image 
from an input digital image, wherein the com- 
pressed digital image is organized into layers hav- 
ing information that can be extracted in accordance 
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with different desired viewing conditions and reso- 
lutions, comprising the steps of: 

(a) decomposing the input digital image to pro- 
duce a plurality of subbands, each subband 
having a plurality of subband coefficients; 

(b) quantizing the plurality of subband coeffi- 
cients of each subband of the decomposed in- 
put digital image to produce a quantized output 
value for each subband coefficient of each sub- 
band; 

(c) forming at least one bit-plane from the quan- 
tized output values of subband coefficients of 
each subband; 

(d) entropy encoding each bit-plane of each 
subband in at least one pass to produce a com- 
pressed bit-stream corresponding to each 
pass, wherein each subband is entropy encod- 
ed independently of the other subbands; 

(e) providing a visual quality table that specifies 
a number of expected visual quality levels by 
providing selectable viewing conditions and 
resolutions for each expected visual quality lev- 
el; 

(f) identifying a minimal set of passes and their 
corresponding compressed bit-streams that 
are necessary to satisfy the expected visual 
quality levels provided in the visual quality ta- 
ble, whereby a user can select different desired 
viewing conditions and resolutions for each 
compressed image; and 

(g) ordering the compressed bit-streams corre- 
sponding to passes into layers from the lowest 
expected visual quality level to the highest ex- 
pected visual quality level specified in the visual 
quality table to produce a compressed digital 
image, wherein each layer includes the passes 
and their corresponding compressed bit- 
streams from the identified minimal set corre- 
sponding to the expected visual quality level 
that have not been included in any lower visual 
quality layers. 

2. The method according to claim 1 further including 
the step of producing a byte-count table that con- 
tains the number of bytes for the compressed bit- 
stream corresponding to each pass of each bit- 
plane of each subband, and wherein the ordering 
step includes using such byte-count table to identify 
the number of bytes sufficient to satisfy each ex- 
pected visual quality level. 

3. The method according to claim 1 wherein the iden- 
tifying step is performed using a model of the con- 
trast sensitivity function for the human visual sys- 
tem. 

4. A computer program product for causing a compu- 



ter to perform the method of claim 1 . 

5. A method for producing a compressed digital image 
from an input digital image, wherein the com- 
s pressed digital image is organized into layers hav- 
ing information that can be extracted in accordance 
with different desired viewing conditions and reso- 
lutions, comprising the steps of: 

10 (a) decomposing the input digital image to pro- 

duce a plurality of subbands, each subband 
having a plurality of subband coefficients; 

(b) quantizing the plurality of subband coeffi- 
cients of each subband of the decomposed in- 

is put digital image to produce a quantized output . 

value for each subband coefficient of each sub- 
band; 

(c) partitioning each subband into a plurality of 
codeblocks; 

20 (d) forming at least one bit-plane from the quan- 

tized output values of subband coefficients of 
each codeblock of each subband; 

(e) entropy encoding each bit-plane of each 
codeblock for each subband in at least one 

25 pass to produce a compressed bit-stream cor- 

responding to each pass, wherein each code- 
block is entropy encoded independently of the 
other codeblocks; 

(f) providing a visual quality table that specifies 
30 a number of expected visual quality levels by 

providing selectable viewing conditions and 
resolutions for each expected visual quality lev- 
el; 

(g) identifying a minimal set of passes and their 
35 corresponding compressed bit-streams that 

are necessary to satisfy the expected visual 
quality levels provided in the visual quality ta- 
ble, whereby a user can select different desired 
viewing conditions and resolutions for each 
40 compressed image; and 

(h) ordering the compressed bit-streams corre- 
sponding to passes from the lowest expected 
visual quality level to the highest expected vis- 
ual quality level specified in the visual quality 

45 table to produce a compressed digital image, 

wherein each layer includes the passes and 
their corresponding compressed bit-streams 
from the identified minimal set corresponding 
to the expected visual quality level that have not 

so been included in any lower visual quality layers. 

6. The method according to claim 5 further including 
the step of producing a byte-count table that con- 
tains the number of bytes for the compressed brt- 
55 stream corresponding to each pass of each bit- 
plane of each subband, and wherein the ordering 
step includes using such byte-count table to identify 
the number of bytes sufficient to satisfy each ex- 
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pected visual quality level. 

7. The method according to claim 5 wherein the iden- 
tifying step is performed using a model of the con- 
trast sensitivity function for the human visual sys- s 
tern. 

8. A computer program product for causing a compu- 
ter to perform the method of claim 5. 

10 

9. A digital camera for capturing an image of a scene 
and for producing a compressed digital image from 
an input digital image, wherein the compressed dig- 
ital image is organized into layers having informa- 
tion that can be extracted in accordance with differ- is 
ent desired viewing conditions and resolutions, 
comprising; 

(a) image sensor means for capturing an image 

of the scene and producing a digital image rep- 20 
resentative of the scene; 

(b) means for processing the digital image to 
provide a processed digital image; 

(c) decomposing the processed digital image to 
produce a plurality of subbands, each subband 25 
having a plurality of subband coefficients; 

(d) quantizing the plurality of subband coeffi- 
cients of each subband of the decomposed dig- 
ital image to produce a quantized output value 

for each subband coefficient of each subband; so 

(e) partitioning each subband into a plurality of 
codeblocks; 

(f) forming at least one bit-plane from the quan- 
tized output values of subband coefficients of 
each codeblock of each subband; 35 

(g) entropy encoding each bit-plane of each 
codeblock for each subband in at least one 
pass to produce a compressed bit-stream cor- 
responding to each pass, wherein each code- 
block is entropy encoded independently of the *o 
other codeblocks; 

(h) providing a visual quality table that specifies 
a number of expected visual quality levels by 
providing selectable viewing conditions and 
resolutions for each expected visual quality lev- « 
el; 

(i) identifying a minimal set of passes and their 
corresponding compressed bit-streams that 
are necessary to satisfy the expected visual 
quality levels provided in the visual quality ta- so 
ble, whereby a user can select different desired 
viewing conditions and resolutions for each 
compressed image; 

(j) ordering the compressed bit-streams corre- 
sponding to passes from the lowest expected ss 
visual quality level to the highest expected vis- 
ual quality level specified in the visual quality 
table to produce a compressed digital image. 



wherein each layer includes the passes and 
their corresponding compressed bit-streams 
from the identified minimal set corresponding 
to the expected visual quality level that have not 
been included in any lower visual quality layers; 
and 

(k) storage means for storing the compressed 
digital image: 
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